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A graphical framework is used for statistical analysis of the results from an extensive re- 
version test of a collection of Reynolds-averaged Navier-Stokes computational fluid 
dynamics codes. The solutions were obtained by code developers and users from North 
America, Europe, Asia, and South America using a common grid sequence and multiple 
turbulence models for the June 2012 fifth Drag Prediction Workshop sponsored by the 
AIAA Applied Aerodynamics Technical Committee. The aerodynamic configuration for this 
workshop was the Common Research Model subsonic transport wing-body previously used 
for the 4 th Drag Prediction Workshop. This work continues the statistical analysis begun in 
the earlier workshops and compares the results from the grid convergence study of the most 
recent workshop with previous workshops. 
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Nomenclature 

aspect ratio 

drag coefficient 

pressure drag coefficient 

skin friction drag coefficient 

lift coefficient 

pitching moment coefficient 

coverage factor for individual values 

Mach number 

number of observations in a sample 

Reynolds number based on reference chord 

value of an observation 

sample mean of a set of observations 

sample median 

population mean 

estimate of the population mean 

population standard deviation 

estimate of the population standard deviation 

A I A 

Coefficient of variation = O / U 


1. Introduction 

I N June 2012, the AIAA Applied Aerodynamics Technical Committee (APATC) sponsored the fifth Drag 
Prediction Workshop (DPW-V) for transonic cruise drag predictions of subsonic transports. The workshop was a 
follow-on to the first Drag Prediction Workshop (DPW-I) held in June 2001, 12 the second Drag Prediction 
Workshop (DPW-II) held in June 2003, 3,4 the third Drag Prediction Workshop (DPW-III) held in June 2006, 5,6 and 
the fourth Drag Prediction Workshop (DPW-IV) held in June 2009 7 ' 8 . The objectives for all five workshops were (1) 


Head, Computational AeroSciences Branch, Mail Stop 128, Associate Fellow AIAA. 

1 

American Institute of Aeronautics and Astronautics 



to assess the state-of-the-art computational methods as practical aerodynamic tools for aircraft force and moment 
prediction of industry relevant geometries, (2) to provide an impartial forum for evaluating the effectiveness of 
existing computer codes and modeling techniques using Reynolds averaged Navier-Stokes solvers, and (3) to 
identify areas needing additional research and development. 

DPW-I solicited CFD predictions of the lift, drag, and pitching moment for the DLR-F4 subsonic transport wing- 
body configuration. The DLR-F4 wing-body configuration 911 was chosen due to public availability of the geometry 
and experimental data from three wind tunnels. Test cases consisted of a single point solution at a fixed value of Cl 
(Cl = 0.500±0.001), calculation of a drag polar, and an optional calculation of drag rise at constant values of Cl. 
Grids were made available for participants, but DPW-I did not require a grid convergence study. A total of 38 
solutions were submitted for the workshop from 18 authors using 13 different CFD codes. A summary of the results 
of DPW-I is given in Ref. 1, and a statistical analysis of the results is given in Ref. 2. The code-to-code statistical 
analysis of the DPW-I results identified two major surprises: 2 (1) roughly 20% of the solutions were statistical 
outliers compared to the others, and (2) the code-to-code scatter for drag was more than an order of magnitude larger 
than desired by airframe designers. 

DPW-II focused on a grid refinement study and the prediction of installed pylon-nacelle drag increments. The 
DLR-F6 12 wing-body and wing-body-nacelle-pylon configurations were chosen for DPW-II since DLR and ONERA 
made data publicly available for this configuration. Test cases consisted of a single point solution at a fixed value of 
Cl (Cl = 0.500±0.001) for both the DLR-F6 Wing-Body and Wing-Body-Pylon-Nacelle configurations on coarse, 
medium, and fine grids, and a drag polar. Optional test cases included a comparison of tripped and fully turbulent 
solutions and calculation of drag rise at fixed values of C L . A total of 21 solutions were submitted for the workshop 
from 20 authors using 18 different CFD codes. There were 16 solutions that calculated all three grid levels for both 
the DLR-F6 Wing-Body and Wing-Body-Pylon-Nacelle from 15 authors using 15 different CFD codes. A summary 
of the results is given in Ref. 3, and a statistical analysis of the results is given in Ref. 4. The DLR-F6 configuration 
had substantial areas of separation at the wing-body juncture and at the wing-pylon juncture. Additionally, there was 
a region of separation at the trailing edge of the wing. The code-to-code statistical analysis of the DPW-II results 
identified four major findings: 4,6 (1) roughly 20% of the grid convergence study solutions were statistical outliers, 
(2) the code-to-code scatter for the wing-body configuration on the medium grid was significantly reduced 
compared to DPW-I, (3) the code-to-code scatter was still significantly larger than desired by airframe designers, 
and (4) there was no significant change in code-to-code scatter with increasing grid density. 

The panel session discussion at the conclusion of DPW-II identified three suggestions for a third workshop: (1) 
the large regions of separation were a likely culprit for the lack of grid convergence therefore cases should be chosen 
with minimal separation, (2) simpler cases were required to allow for better grid convergence studies and wider 
participation, and (3) blind studies were preferable where experimental data were not available a priori. It was 
generally agreed that continuing studies of the DLR-F6 case were warranted. Vassberg et al. 13 designed a side-of- 
body fairing to produce attached flow in the wing-body juncture. The fairing was designated the FX2B and the 
configuration was referred to as the DLR-F6-FX2B. An experimental program at DLR and NASA collected data for 
the DLR-F6 with and without the FX2B fairing. 14 The data were not collected before the workshop and the 
workshop was run blind. Additionally, two isolated wings, designated DPW-W1 and DPW-W2, were designed 5 for 
a simple geometry with DPW-W2 a single point optimization of DPW-W1. 

DPW-III focused on grid convergence studies and predicted increments for the DLR-F6 wing-body with and 
without the FX2B side-of-body fairing. The Reynolds number was increased to 5 million to minimize the trailing 
edge separation. The choice of Reynolds number was limited by stress analysis of the wind tunnel model. Additional 
test cases included a drag polar at fixed Mach number and an optional Reynolds number scaling study. An optional 
Case 2 grid convergence study included four grid levels for the DPW-W1 and DPW-W2 isolated wing cases and a 
drag polar at a fixed Mach number. Case 2 specified the angle-of-attack rather than a fixed Cl- A summary of the 
DPW-III results is given in Ref. 5, and a statistical analysis is given in Ref. 6. The code-to-code statistical analysis 
of DPW-III identified four major findings: 6 (1) roughly 20% of the solutions for the grid convergence study of the 
Case 1 DLR-F6 wing-body configuration were statistical outliers; none of the solutions for the Case 2 wing alone 
were statistical outliers, (2) the code-to-code scatter for Case 1 DLR-F6 wing-body fine grid showed no 
improvement over DPW-II, (3) the code-to-code scatter was still significantly larger than desired by airframe 
designers, and (4) the code-to-code scatter showed a reduction with increasing grid resolution when the variation in 
lift was appropriately accounted for. However, the code-to-code scatter for DPW-III was larger for the coarse and 
medium grids than for DPW-II. 

DPW-IV focused on a grid convergence study and a downwash study including prediction of trimmed drag on 
the NASA Common Research Model (CRM) wing-body-horizontal tail configuration. Optional cases included a 
Mach sweep study at fixed C L and a Reynolds number study. The CRM was a new wing-body-horizontal tail 
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configuration, with and without nacelle-pylons, developed by the NASA Subsonic Fixed Wing (SFW) 
Aerodynamics Technical Working Group (TWG) in collaboration with the DPW Organizing Committee. 15 The 
CRM is representative of a contemporary high-performance transonic commercial transport. DPW-IV was a true 
blind test; the workshop was held in June 2009 and all the data submittals were completed before experimental data 
were collected. Experimental data were collected in the NASA Langley National Transonic Facility in January- 
February 2010 and in the NASA Ames 1 1 -ft wind tunnel during March-April 2010. 16-17 A summary of the results 
from DPW-IV is provided in Ref. 7. Detailed results from participants are provided in Refs. 18-30. 

The code-to-code statistical analysis of DPW-IV identified five major findings: 8 (1) roughly 12.5% of the 
solutions for the Case 1 grid convergence study were outliers, (2) the code-to-code scatter for the total drag, 
normalized by the total drag to account for different drag levels between cases, was virtually the same for DPW-II, 
DPW-III, and DPW-IV, (3) the code-to-code variation of the forces and pitching moment, normalized by the 
appropriate force or pitching moment, was substantially larger on the horizontal tail component than on the wing or 
fuselage component, (4) the code-to-code variation was still substantially higher than desired by airframe designers, 
and (5) the drag (total drag, pressure drag, and skin friction drag) showed some reduction in scatter with increasing 
grid resolution. 

The panel discussion at the conclusion of DPW-IV recommended continuing the workshop series with 
continuing focus on the CRM and a grid convergence study using a common set of grids for all CFD codes. DPW-V 
focused on a grid convergence study on the CRM wing-body using a Common Grid Sequence. Two optional cases 
included a wing-body buffet study at fixed Mach and a turbulence model verification study. Vassberg 31 generated a 
five block multi-block grid around the CRM wing-body. He generated six grid levels ranging from approximately 
640 thousand grid points on the coarsest grid to over 138 million grid points on the finest mesh. This multi-block 
grid was converted to overset, unstructured hexahedral, unstructured prisms, and unstructured hybrid grids using the 
exact same set of grid points. The grid convergence study required at least four of the six grid levels. Optionally, 
participants could include solutions on grids they developed. The wing-body buffet study used the medium grid 
from the Common Grid Sequence at seven angles of attack in the region of the flow where substantial separation 
developed. This paper analyzes the statistics of the Case 1 grid Convergence Study and compares it to earlier grid 
convergence results. 

This paper is organized in the following manner. Section II provides a description of the statistical analysis. 
Section III outlines the test cases for the workshop. Section IV details the statistical results. Section V provides some 
summary comments. 


II. Statistical Approach 

Hemsch 2 introduced the idea of treating different computations of a test case as a collective and using N-version 
testing in a statistical framework to investigate the submissions. No individual result is considered the “right” or 
“best” result. This framework is useful for identifying differences between submissions. The dispersion of the results 
is treated as noise in the collective computational process. 

A running record of individual outcomes is plotted for each of the measures of interest and derived quantities 
reported by participants. Participants reported the angle of attack (a), total drag coefficient (C D ), pressure drag 
coefficient (C D/W )> skin friction drag coefficient (C D ^), idealized profile drag, and moment coefficient ( C m ). The 

value of the measure of interest is plotted on the vertical axis and an integer index is used for each data submission 
on the horizontal index. The order of the solutions on the horizontal axis is irrelevant since this is not a temporal 
axis. 

An estimate of the population mean jj, of the plotted data submissions is made and is shown on the graph as the 
centerline. Upper and lower scatter limits are placed on the graph as follows: 

Upper Limit = /t + Kef (1) 

Lower Limit = il — Ko (2) 

where 0 is an estimate of the population standard deviation and K is an appropriate coverage factor. Significant 
results are outcomes that lie outside the process limits defined in Eqs. (1) and (2). These results, referred to as 
outliers, represent submissions that are different from the results that lie within the scatter limits (and should be 
investigated to understand the difference). 
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The population mean fi is estimated using the sample median, which is given (for sorted data) as: 


il = X 

X s X (n + l)l2 (n odd ) (3) 

X = 0.5(x n/2 + -^( n/ 2 ) +1 ) (n even) 

The sample median provides a robust estimate when outliers are present. The sample standard deviation 


a = SSD = 



(4) 


is used to estimate the population standard deviation. The coverage factor is estimated for a uniform distribution 32 as 
K = S . This value of coverage factor was chosen to look for differences in submissions. 


III. DPW-V Test Cases 

The geometry for DPW-V was the CRM wing-body as described in Vassberg et al. 15 and shown in Fig. 1. Case 1 
consisted of a grid convergence study with the Common Grid Sequence and with optional participant developed 
grids. The Case 2 Wing-Body Buffet Study and Case 3 Turbulence Model Verification Study were each optional 
and are not considered here. All simulations were specified to be in free air, i.e. no wind tunnel walls or model 
support systems were to be included. Boundary layers were to be modeled as fully turbulent, i.e. no transition 
location was specified. The moment reference center is x REF = 1,325.90 in, z REF = 177.95 in. Additional reference 
quantities are: S REF = 594,720 in 2 ; c^p = 275.80 in; and AR = 9.0. 

The primary focus of DPW-V was a grid convergence study using the Common Grid Sequence. Vassberg 31 first 
generated the finest grid, denoted SuperFine (L6), on the CRM wing-body using five blocks for a multi-block grid, 
Fig. 2. He then coarsened the SuperFine grid by a factor of 3:2 in each coordinate direction to create the ExtraFine 
(L5) grid. He coarsened the SuperFine grid by deleting every other grid line in each coordinate direction to generate 
the Fine (L4) grid and repeated the process to create the Coarse (L2) grid. He deleted every other grid line in each 
coordinate direction from the ExtraFine (L5) grid to generate the Medium (L3) grid and repeated the process to 
generate the Tiny (LI) grid. The six grid levels range in size from approximately 639 thousand grid points to over 
138 million grid points, see Table 1. All other common grids use these same grid points. The Common Grid 
Sequence was made available to participants in multi-block, overset, unstructured hexahedral, unstructured prisms, 
and unstructured hybrid (prisms in the boundary layer and tetrahedron outside the boundary layer) forms. A 
summary of the grids and results is provided in Ref. 33. 

The three cases are summarized below. 


Table 1. Multi-block grid family. 


Level 

Name 

Hexahedra 

Y+ 

# constant cells 

LI 

Tiny 

638,976 

2.00 

2 

L2 

Coarse 

2,156,544 

1.33 

3 

L3 

Medium 

5,111,808 

1.00 

4 

L4 

Fine 

17,252,352 

0.67 

6 

L5 

ExtraFine 

40,894,464 

0.50 

8 

L6 

SuperFine 

138,018,816 

0.33 

12 


A. Case 1 - NASA Common Research Model (CRM) Wing-Body Common Grid Study: 

Case 1 focused on a grid convergence study of the CRM Wing-Body. The Reynolds number was chosen to be 5 
million to match the experimental data: 

1) Grid convergence study at Mach = 0.85, Cl = 0.500 (±0.001) 
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• Grid refinement series from the Common Grid Sequence consisting of at least four grid levels. Target 

grids should range from 3 to 50 million unknowns. 

• Chord Reynolds number 5 x 1 0 5 6 based on c REF 

• Reference T emperature = 1 00° F 

2) Optional Grid Convergence study using participant developed grids 

• All participants are encouraged to build their own grids using ‘best practice’ techniques. IGES and 

CATIA models of the 1 -g shape at the design point are available for grid construction. A family of grid 
sizes should be developed in the range described above suitable for Grid Convergence Study. All grids 
used for results presented at the workshop must be submitted to the DPW Organizing Committee to be 
made available to all interested parties. Results and grids will be published electronically on the DPW 
web site. 

B. Case 2 (Optional) - NASA Common Research Model (CRM) Wing-Body Buffet Study: 

Calculate Drag Polars at: 

• Mach = 0.85 

• Drag Polar for a = 2.50°, 2.75°, 3.00°, 3.25°, 3.50°, 3.75°, 4.00° 

• Medium grid used in Case 1 from the Common Grid Sequence or participant developed grids 

• Chord Reynolds number 5 x 1 0 6 based on Cre F 

• Reference Temperature = 100°F 

C. Case 3 (Optional) - Turbulence Model Verification: 

Details for the following three cases are listed on the TMBWG website. 34 For each case, use the three finest 
supplied grids. 

1 . 2D Zero Pressure Gradient Flat Plate 

2. 2D Bump-in-channel 

3. 2D N AC A00 12 Airfoil 


IV. Results 

The data were submitted to the DPW committee before the workshop. After the workshop, authors were given 
time to evaluate and resubmit their solutions. Several authors took advantage of this time and submitted corrections 
to their solutions or provided additional data that they did not have time to complete before the workshop. 

A. Case 1 NASA CRM Wing-Body Common Grid Study 

For the Case 1 CRM Wing-Body Common Grid Study: 

1. There were 16 different CFD codes used 

2. There were 17 organizations that participated 

3. There were 44 solutions submitted: 

a. 37 solutions on the common grid system and 

b. 7 solutions on custom grids; 

4. Of the 37 solutions on the Common Grids, 

a. 7 solutions used the Common Multiblock grid, 

b. 5 solutions used the Common Overset grid, 

c. 14 solutions used the Common Hex grid, 

d. 4 solutions used the Common Prism grid, and 

e. 7 solutions used the Common Hybrid grid. 

5. There were 5 turbulence models used (not counting variants of basic models): 

a. 27 of the 44 solutions from 15 participants used the one-equation Spalart-Allmaras model 
(several variants were reported); 

b. 4 solutions from one participant used the one-equation RT model; 

c. 1 1 solutions from 6 participants used the two-equation SST model; 

d. 1 solution used an explicit algebraic Reynolds stress model (EARSM) model; and 

e. 1 solution used the lagRST Reynolds stress closure 

f. Some authors reported variants of the basic models that they used. 

6. All 44 submissions were provided on three of the grid levels - Coarse (L2), Medium (L3), and Fine (L4). 


5 

American Institute of Aeronautics and Astronautics 



Nested solutions are defined as the solutions that provided data for all of the required grid levels. For previous 
Drag Prediction Workshops, not all of the submissions provided data on all levels for the grid convergence case. For 
DPW-V, four grid levels were required. All 44 DPW-V Case 1 submissions provided solutions on the L2, L3, and 
L4 grids. Some of the participants added the LI grid to make the four grid levels and some added the L5 grid to 
make the requirement. The nested solutions for DPW-V are defined to be all 44 submissions since they provided 
solutions on a common set of three grid levels. 

The statistical analysis below identifies outliers that are different from the other solutions. The core solutions are 
the solutions that are similar to each other, i.e. the core solutions are the nested solutions minus the outliers. Table 2 
summarizes the number of submissions, nested submissions, and core submissions for DPW-II through DPW-V. 
The number of submissions and the number of solutions in the core is larger for DPW-V than any of the earlier 
workshops. The number of authors in the core, the number of institutions, and the number of codes is higher for 
DPW-V than earlier workshops. 

Table 2. Comparison of workshop statistics for grid convergence case in DPW-II through DPW-V. 



DPW-II 

DLR-F4 

Wing-Body 

DPW-III 

DLR-F6 with FX2B 
fairing 
Wing-Body 

DPW-IV 

CRM 

Wing-Body-Horizontal 

Tail 

DPW-V 

CRM 

Wing-Body 

Sub. 

Nested 

Core 

Sub. 

Nested 

Core 

Sub. 

Nested 

Core 

Sub. 

Core 

Solutions 

21 

16 

13 

26 

20 

16 

29 

24 

21 

44 

37 

Authors 

20 

15 

12 

15 

12 

9 

17 

16 

13 

20 

19 

Institutions 

16 

14 

11 

13 

10 

9 

15 

14 

11 

17 

16 

Codes 

18 

15 

12 

14 

12 

8 

16 

15 

12 

16 

15 


1. Individual Charts for Case 1 Grid Convergence Study 

Running records of individual outcomes were plotted for each of the measures of interest for each grid level in 
the grid resolution study. The population mean, estimated by the median calculated from Eq. (3), is shown as a solid 
black line and the scatter limits calculated from Eq. (1) are shown as dashed black lines. In all of the following plots 
for Case 1, the solution index on the abscissa refers to the same solution submission. Figures 3-5 show the 5 
measures of interest for the Coarse (L2), Medium (L3), and Fine (L4) grid levels, respectively. The Tiny (LI), 
ExtraFine (L5), and SuperFine (L6) grid levels have fewer submissions; therefore the estimate of variation is less 
accurate. LI, L5, and L6 show similar trends for L2-L4 for solutions that were submitted. 

Five solutions are outside the limits for the total drag (Figs. 3a, 4a, 5a) and the pressure drag (Figs. 3b, 4b, 5b) 
for grid levels L2-L4: Solutions 6, 8, 10, 40 and 41. Solution 37 is significantly above the limits for skin friction 
drag (Figs. 3c, 4c, 5c) on all five submitted grid levels (L1-L5). 

For angle of attack, solution 8 is outside the limits for the Fine (L4) grid (Fig. 5d). For the pitching moment 
coefficient, solution 8 is outside the limits on Medium (L3) and Fine (L4) grid levels (Figs. 4e, 5e). 

2. Graphical Evaluation of Grid Convergence for Case 1 

In order to compare results on unstructured meshes with results on multi-block structured meshes and overset 
structured meshes, it is convenient to use a one-dimensional estimate of the grid spacing, which is given for three 
dimensions as 


Mnpts 

where NPTS is the number of solution points (for either cell-centered or vertex based CFD codes) in the mesh. 
Second order accurate results in the asymptotic range will lie on a straight line when plotted versus h 2 (NPTS' 2 3 ). 
An additional benefit of plotting versus NPTS' 2,3 is that coarse meshes show up on the right of the plot and finer 
grids approach the y-axis, which is the limit of infinite grid resolution. 
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Salas 35,36 has shown that the error is linear with h p (where p is the order of accuracy of the scheme) only if the 
grid is refined uniformly in all coordinate directions, e.g. if the grid spacing is halved in the x coordinate, it must 
also be halved in the y and z coordinates. The Common Grid Sequence grids are uniformly refined in all coordinate 
directions. However, it was not verified that participant generated grids met this requirement. Other factors can also 
contribute to this non-linearity. 

It is difficult to see the grid related trends in the submissions looking at the running records. Therefore the total 
drag and the pitching moment coefficient for all of the submissions for Case 1 are plotted in Figs. 6 and 7. These 
figures include the estimate of the mean shown as a solid black line and the limits shown as dotted black lines. The 
lower scatter limits for the LI, L5, and L6 grids is a result of reduced number of submissions and is not a good 
comparison to the scatter on grids L2, L3, and L4. (The scatter on the core solutions is more consistent, see Figs. 8- 
1 1 .) Multi-block grid solutions are plotted with green lines and symbols, overset grid solutions are plotted with red 
lines and symbols, unstructured hybrid grid solutions are plotted with solid blue lines and symbols, unstructured 
hexahedral grid solutions are plotted with dashed blue lines and symbols, and unstructured prism grids are plotted 
with dash-dot blue lines and symbols. Solutions on participant generated custom grids are plotted with brown lines 
and symbols. In addition to Solutions 6, 8, 10, 37, 40, and 41 that were identified in the individual charts as outside 
the limits of the solutions, Solution 9 is also outside the limits of the solutions. 

3. Analysis of Outliers 

The previous two sub-sections identified solutions 6, 8, 9, 10, 37, 40, and 41 as outliers (approximately 16% of 
the submissions). 

Solutions 6, 8, 9, 10, 40 

Solutions 6, 8, 10, and 40 are outside the limits for all three submitted grid levels (L2, L3, and L4) for both total 
and pressure drag; demonstrating higher predicted total and pressure drag than the collective. The skin friction drag, 
angle of attack, and moment coefficient are inside the limits for these solutions on the same grid levels. Solutions 8 
and 40 are significantly outside the limits while solutions 6 and 10 are closer to the limits. 

Solutions 6, 8, 10, and 40 are four of eight submissions from the same team. All of the submissions from this 
group used the same solver. The eight submissions correspond to two different turbulence models (RT and SST) and 
four different grids (Common Hex, Common Hybrid, Common Prism, and Custom). Solutions 6, 8, 10, and 40 all 
used the RT turbulence model. Solutions 7, 9, 11, and 41 used the SST turbulence model. 

Solution 32 from another participant using the same solver and the Common Prism grid as solution 10 but using 
the SA-RC variant of the SA turbulence model, was within the limits for all the grid levels submitted and all the 
figures of merit. Solution 1 1 was from the same participant as solution 1 0, used the Common Prism grid and the 
SST turbulence model and was within the limits for all grid levels submitted and all figures of merit. Solution 7 on 
the Common Hex grid was within limits and Solution 9 on the Common Hybrid grid was outside the limits; both 
used the SST turbulence model. Since the flow solver with SST or SA-RC turbulence model on two of the three 
grids gives results within the limits for all figures of merit, the difference between solutions 6, 8, 10, and 40 and the 
collective can be attributed to the RT turbulence model. At the workshop, the presenter of these solutions noted that 
the RT model predicted a larger trailing edge separation and a larger side-of-body separation than the SST 
turbulence model. This is consistent with the larger total and pressure drag coefficient seen in the submitted data. 
Since the only difference between solutions 5, 7, and 9 is the grid, the difference between solution 9 and the 
collective can be attributed to the use of the hybrid grid and could be due to the numerical differences on hybrid 
grids. 

Solution 41 

Solution 41 is outside the limits (high) for total and pressure drag on the all three submitted grid levels (L2, L3, 
L4) (Figs. 3b, 4b). The solutions get better relative to the collective as the grid is refined. Solution 41 is from the 
same participant group that submitted solutions 6-11 and 40. Solution 41 used the SST turbulence model and a 
custom grid. Solution 40 and 4 1 report higher total and pressure drag for the respective turbulence models than the 
solutions on the Common Hex, Common Hybrid, or Common Prism grids (compare solution 40 to solutions 6, 8, 
and 10; compare solution 41 to solutions 7 and 11). Solutions 40 and 41 use the same code as their other 
submissions. Therefore, the higher value of total and pressure drag across the grid levels can be attributed to this 
custom grid. 
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Solution 37 

Solution 37 is considerably high for the skin friction drag coefficient for all five submitted grid levels (L1-L5). 
However, all other measures of interest (angle-of-attack, total drag, pressure drag, skin friction drag) are well within 
the limits of the collective. Solution 37 used the SA turbulence model used by many other authors that were within 
the limits of the collective. Solution 37 used the Common Multiblock mesh that was used several other participants. 
At the workshop, this participant group showed iterative convergence of approximately nine orders of magnitude for 
the density residual on all grid levels. Therefore iterative convergence does not appear to be the cause of high values 
of skin friction. 

During the workshop, several participants identified an issue with skin friction. One issue was in the definition of 
the wall normal distance used in various turbulence models. Some participants noted that they calculated the wall 
normal distance by integrating along a grid line. This is inaccurate for wall distance when the grid is skewed near the 
surface as the Common Grid Sequence is. Other participants noted that their formula for calculating wall shear stress 
was less accurate for skewed grids. Several participants provided updated results correcting these issues after the 
workshop. It is possible that this participant has a similar issue. 

4. Core Solutions 

Since solutions 6, 8, 9, 10, 37, 40, and 41 exhibit substantially different behavior from the rest of the collective 
over a range of grids, not just on a single grid, they are removed from the collective and are not part of the core 
solutions. The core solutions consist of the other 37 solutions. 

The median values for the Case 1 Grid Convergence Study core solutions at three grid levels is given in Table 3. 
Table 4 shows the estimate of the standard deviations for the core solutions for the Case 1 Grid Convergence Study. 
Figures 8 through 1 1 show the variation of the core solutions including new estimates of the median and scatter 
limits for the total drag, pressure drag, skin friction drag, and the pitching moment coefficients, respectively. These 
figures include the estimate of the mean for the core solutions shown as a solid black line and the limits of the core 
solutions shown as dotted black lines. Multi-block grid solutions are plotted with green lines and symbols, overset 
grid solutions are plotted with red lines and symbols, unstructured hybrid grid solutions are plotted with solid blue 
lines and symbols, unstructured hexahedral grid solutions are plotted with dashed blue lines and symbols, and 
unstructured prism grids are plotted with dash-dot blue lines and symbols. Solutions on participant generated 
custom grids are plotted with brown lines and symbols. The core solutions for the total drag and drag components 
show a converging trend as the grid is refined. However, the pitching moment shown in Fig. 11 does not show a 
converging trend with grid refinement. 


Table 3. Comparison of Case 1 Grid Convergence Study core medians. 



Tiny (LI) 

Coarse (L2) 

Medium (L3) 

Fine (L4) 

ExtraFine 

(L5) 

SuperFine 

(L6) 

a, degrees 

2.293 

2.234 

2.220 

2.200 

2.187 

2.178 

C D , counts 

267 

256 

253 

251 

250 

250 

C DPR , counts 

158 

144 

141 

137 

136 

136 

Cdsf> counts 

111 

112 

113 

114 

114 

114 

C M 

-0.1080 

-0.1117 

-0.1128 

-0.1143 

-0.1156 

-0.1157 


Table 4. Comparison of Case 1 Grid Convergence Study core standard deviations. 



Tiny (LI) 

Coarse (L2) 

Medium (L3) 

Fine (L4) 

ExtraFine 

(L5) 

SuperFine 

(L6) 

a, degrees 

0.057 

0.046 

0.045 

0.047 

0.048 

0.052 

C D , counts 

9.8 

4.6 

3.2 

2.7 

2.3 

2.1 

Cdpr, counts 

8.7 

4.4 

3.3 

3.0 

2.7 

2.3 

Cdsf, counts 

3.7 

2.5 

2.0 

1.7 

1.7 

1.5 

Cm 

0.0041 

0.0039 

0.0040 

0.0042 

0.0045 

0.0049 
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B. Comparison of DPW-V results with DPW-II, DPW-III, and DPW-IV 

DPW-II, DPW-III, and DPW-IV each included a grid convergence study. Gridding guidelines were posted 
before each workshop and included recommended sizes for each of the grids in the grid convergence studies to 
provide a guide to the participants. Grid sizes tripled over the course of these three workshops. However, the 
questions remains whether increasing grid sizes have improved the results. 

The coefficient of variation, 37 C v (= a/p), provides a measure to compare the variation of populations with 
different means. The results of DPW-V are compared to the results of DPW-II, 4 DPW-III, 6 and DPW-IV 8 for the 
cases that were run at constant lift. Morrison and Hemsch 6 showed that the variation in lift affected the comparisons 
of drag quantities and pitching moment and showed that quantities that corrected to a constant lift condition, e.g. 
L/D and idealized profile drag, were required to compare results at different values of lift. 

The coefficient of variation for the pitching moment coefficient would be negative since the mean is negative 
and the estimate of the standard deviation is positive. Therefore, |C V | is plotted for the pitching moment coefficient to 
make comparisons easier. 

The coefficients of variation for the total drag, pressure drag, and skin friction drag coefficients are shown in 
Figs. 12 through 14 for DPW-V and earlier workshops. The number of core solutions varies by grid level: LI had 28 
solutions, L2 - L4 had 37 solutions, L5 had 29 solutions, and L6 had only 13 solutions. The variation of the 
statistics decreases as the number of samples decreases. An estimate was made of the variation of the statistics 37 for 
the DPW-V results that had varying sample sizes for each grid level. This estimate of variation in C v is shown as 
dotted lines surrounding the estimate of C v . The variation in C v is almost twice as large for the L6 grid as the L2 - L4 
grids. There are two important trends to note for C v of the forces. First, the DPW-V variation for all drag measures 
decreases from the coarsest mesh to the finest mesh. Second, the DPW-V variation for all drag measures is lower 
than earlier workshops (DPW-II through DPW-IV). This variation includes different turbulence models, different 
codes and numerics, different participants, different forms of the Common Grid Sequence grids, and even a few 
custom grids. None of the previous workshops included as wide range of grid sizes as DPW-V and none of them 
achieved as low a value of variation or such a consistent trend. The number of participants represented in the core 
solutions was larger for DPW-V than the earlier workshops. The number of codes and turbulence models was at 
least as large in DPW-V as earlier workshops. It appears that the Common Grid Sequence achieved its goal of 
improving the variation. 

The coefficient of variation of the pitching moment coefficient, Fig. 15, is also smaller for DPW-V than earlier 
workshops. However, the variation increases slightly on the finest meshes. DPW-IV wing-body-tail results showed 
substantially larger divergence of the variation with grid convergence. It is possible that the increase in variation on 
finer grids for pitching moment is due to larger differences in predicted separation region between turbulence 
models on the more resolved meshes. Additional analysis is required to confirm this. 

Both DPW-V and DPW-IV included optional fields to collect force and moment data on vehicle components 
(wing and fuselage for DPW-V; wing, fuselage, and horizontal tail for DPW-IV). Figures 16-18 (note logarithmic 
scale) show the coefficient of variation for lift coefficient, total drag coefficient, and pitching moment coefficient by 
vehicle component. While Case 1 is a constant lift case, the percentage of lift by component can change. In Fig. 16, 
it is clear that the variation of lift on the wing is substantially lower than the variation for the fuselage for DPW-V. 
The variation of the lift on the wing is lower for DPW-V than DPW-IV. However, the variation of lift coefficient is 
almost the same on the fuselage between DPW-V and DPW-IV. Note that the variation on the tail is higher than 
other components for all forces and moment. The difference between DPW-V and DPW-IV on the wing lift 
coefficient and lack of similar difference for the fuselage could be an indicator that the grid on DPW-V better 
resolves the wing and that the grid on the fuselage is similar between the two workshops. Other causes should be 
investigated as well. The increase in the variation on the finest grids could be differences in predicted separation 
region that increase with grid refinement. The variation in drag coefficient and pitching moment coefficient is very 
similar for each component of DPW-IV and DPW-V. 

V. Concluding Remarks 

A statistical analysis applied to the results of the fifth Drag Prediction Workshop demonstrated that the code-to- 
code scatter of the total drag, pressure drag, and skin friction drag was substantially lower than earlier workshops 
and decreased with increasing grid resolution. However, the code-to-code scatter in the pitching moment did not 
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show a convergent trend as it increased with increasing grid resolution. Additional analysis will be required to 
identify the cause of the lack of converging trend for the pitching moment. 

The statistical analysis identified approximately 16% of the solutions that were outliers compared to the other 
solutions. This has changed by only a small amount from the first workshop through the fifth workshop. Some of 
these outliers are due to code-to-code bias (e.g. different turbulence models will predict different results). 
Continuing efforts are required to understand code-to-code bias. 

The level of code-to-code scatter was reduced for the fifth workshop compared to the second through the fourth 
workshop. The one standard deviation level of scatter is below 1% out of approximately 250 drag counts for total 
drag, or about 2.5 drag counts. While still above the scatter that designers would like to see, this is an improvement 
over earlier workshops. It appears that the Common Grid Sequence has helped to reduce code-to-code scatter by 
removing one cause of scatter. Turbulence model is a source of bias that won’t be reduced by grid resolution. 

The recommendations from DPW-IV for future workshops 8 can be updated: 

1 . Analyze grids (check resolution of the grids on the fuselage and tail in addition to the wing, verify that grids 

meet the gridding guidelines, verify that grids are a family, evaluate grid requirements for codes on simpler 
problems, etc.); grids were sited as a large source of uncertainty in DPW-I through DPW-IV. It appears that 
consistently constructed grids can reduce one source of scatter. 

2. Report iterative convergence results. This was used to understand some sources of variation. 

3. Use common nomenclature 38 when reporting the turbulence model. This is still not universally applied; 

variants in base turbulence models such as SA will give different results. 

4. Do verification tests of turbulence models 38 before applying them to complex flows. There was an effort in 

DPW-V to do some verification tests, but most participants did not run a grid convergence for the turbulence 
model verification cases. 
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Figure 1. Common Research Model geometry (wing-body). 
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Figure 2. Block layout for common multi-block grid for CRM. 
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Figure 3. Coarse (L2) grid measures of interest for Case 1 Grid Convergence Study. 
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Figure 4. Medium (L3) grid measures of interest for Case 1 Grid Convergence Study. 
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Figure 5. Fine (L4) grid measures of interest for Case 1 Grid Convergence Study. 
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Figure 6. Grid convergence of total drag coefficient including estimate of the mean and limits for all 
Case 1 submissions (median - solid black line; limits - dashed black lines; multi-block - green lines; overset - 
red lines; unstructured hybrid - solid blue lines; unstructured hex - dashed blue lines; unstructured prism - 
dash-dot blue lines; custom - dark red). 
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Figure 7. Grid convergence of pitching moment coefficient including estimate of the mean and limits for all 
Case 1 submissions (median - solid black line; limits - dashed black lines; multi-block - green lines; overset 

- red lines; unstructured hybrid - solid blue lines; unstructured hex - dashed blue lines; unstructured prism 

- dash-dot blue lines; custom - dark red). 
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Figure 8. Grid convergence of total drag coefficient including estimate of the mean and limits for core 
submissions for Case 1 (median - solid black line; limits - dashed black lines; multi-block - green lines; 
overset - red lines; unstructured hybrid - solid blue lines; unstructured hex - dashed blue lines; 
unstructured prism - dash-dot blue lines; custom - dark red). 
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Figure 9. Grid convergence of pressure drag coefficient including estimate of the mean and limits for core 
submissions for Case 1 (median - solid black line; limits - dashed black lines; multi-block - green lines; 
overset - red lines; unstructured hybrid - solid blue lines; unstructured hex - dashed blue lines; 
unstructured prism - dash-dot blue lines; custom - dark red). 
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Figure 10. Grid convergence of skin friction drag coefficient including estimate of the mean and limits for 
core submissions for Case 1 (median - solid black line; limits - dashed black lines; multi-block - green lines; 
overset - red lines; unstructured hybrid - solid blue lines; unstructured hex - dashed blue lines; unstructured 
prism - dash-dot blue lines; custom - dark red). 
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Figure 11. Grid convergence of pitching moment coefficient including estimate of the mean and limits for 
core submissions for Case 1 (median - solid black line; limits - dashed black lines; multi-block - green lines; 
overset - red lines; unstructured hybrid - solid blue lines; unstructured hex - dashed blue lines; unstructured 
prism - dash-dot blue lines; custom - dark red). 
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Figure 12. Coefficient of variation of the total drag coefficient (dotted lines represent variation in C v 
estimate). 
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Figure 13. Coefficient of variation of the pressure drag coefficient (dotted lines represent variation in C v 
estimate). 
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Figure 14. Coefficient of variation of the skin friction drag coefficient (dotted lines represent variation in C v 
estimate). 
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Figure 15. Coefficient of variation of the pitching moment coefficient (dotted lines represent variation in C v 
estimate). 
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Figure 16. Coefficient of variation of the lift coefficient for each component of the CRM for DPW-IV (wing, 
tail, fuselage) and DPW-V (wing, fuselage). 
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Figure 17. Coefficient of variation of the drag coefficient for each component of the CRM for DPW-IV (wing, 
tail, fuselage) and DPW-V (wing, fuselage). 
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Figure 18. Coefficient of variation of the pitching moment coefficient for each component of the CRM for 
DPW-IV (wing, tail, fuselage) and DPW-V (wing, fuselage). 
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